Endpoints for task comments #1171

varun2948 · 2024-02-06T10:35:55Z

What type of PR is this? (check all applicable)

Related Issue

Fixes Add support for comments #241

Describe this PR

Integrated a task_comment with migration in backend with field included
id
task_id
project_id
comment_text
commented_by
created_at

Screenshots

Checklist before requesting a review

📖 Read the FMTM Contributing Guide: https://github.com/hotosm/fmtm/blob/main/CONTRIBUTING.md
📖 Read the FMTM Code of Conduct: https://github.com/hotosm/fmtm/blob/main/CODE_OF_CONDUCT.md
👷‍♀️ Create small PRs. In most cases, this will be possible.
✅ Provide tests for your changes.
📝 Use descriptive commit messages.
📗 Update any related documentation and include any relevant screenshots.

Can you check this PR for task_comment @spwoodcock ?

[optional] What gif best describes this PR or how it makes you feel?

…k_comment

for more information, see https://pre-commit.ci

…k_comment

…ature/task_comment

spwoodcock · 2024-02-06T11:06:48Z

This looks like good work, but I'm not sure we need a new table for comments.
I think the idea of the task_history table is that is it's used for various events, including comments (this is what tasking manager does anyway).
See #241 (comment)

Options for the taskaction enum, which is used in task_history:

CREATE TYPE public.taskaction AS ENUM (
    'RELEASED_FOR_MAPPING',
    'LOCKED_FOR_MAPPING',
    'MARKED_MAPPED',
    'LOCKED_FOR_VALIDATION',
    'VALIDATED',
    'MARKED_INVALID',
    'MARKED_BAD',
    'SPLIT_NEEDED',
    'RECREATED',
    'COMMENT'
);

I am open to the idea, but don't want to deviate from the TM schema very much (especially considering Robs work on tm-admin).

varun2948 · 2024-02-06T11:11:08Z

yeah @spwoodcock i had worked even before the comment of rob and yours, so i will change this to the way TM is handling.

spwoodcock · 2024-02-06T11:14:50Z

So sorry for the extra work 🙏
I'm sure most of the code can be adapted, just without the migration.

varun2948 · 2024-02-06T11:20:54Z

no worries sam, will have it changed and yeah the code will work based on that change with little refactor

spwoodcock · 2024-02-06T11:23:51Z

Nice work writing the raw SQL by the way 👍

There are pydantic models in task_schemas for TaskHistory that could be used for input (they may need to be tweaked).
But for the comment output/display, we could use the models you made with a little workaround:

class TaskCommentRequest(BaseModel):
    """Task mapping history."""

    action_text: Any = Field(exclude=True)

    task_id: int
    project_id: int
    comment: Optional[str] = None

    @field_serializer("comment")
    def convert_action_to_comment(self, value: str) -> Optional[str]:
        """Get the task history comment."""
        if self.action_text:
            return self.action_text
        return None

This way when you create the pydantic model from the database (sqlalchemy) model, the action_text will be included in the model, but the actual serialised JSON will not include action_text (but comment instead`).

Not saying we have to do it this way, just giving an option 😉

varun2948 · 2024-02-06T11:34:03Z

Hmm, yeah we can do that but one query is about the task action is, are we supposed to go with comment being enable only if the task gets locked for mapping or any other status, i mean aren't we supposed to give comment based on any status going on like i should be able to comment on the task even before it gets released for mapping, locked for mapping, locked for validation ?

varun2948 · 2024-02-13T04:06:19Z

Fixed linting issue

spwoodcock

Looks very nice!

I will rebase (to fix the failing test), remove the db model, and mark ready.
Merge when you are happy with it👍

src/backend/app/db/db_models.py

src/backend/app/tasks/tasks_routes.py

spwoodcock · 2024-02-13T09:46:55Z

I just made some small updates:

Fixed the linting.
Improved the SQL readability with indenting & by removing the CTE from the insert query.
Used query params to insert vars, instead of f-strings. This is important to prevent possible SQL injection risk.

Before:

    query = text(
        f"""
        WITH inserted_comment AS (
        INSERT INTO task_history
        (project_id,task_id,"action",action_text,action_date,user_id)
        VALUES({comment.project_id},{comment.task_id},'COMMENT',
        '{comment.comment}','{currentdate}',{user_data.id})
        RETURNING
        task_history.id,task_history.task_id, task_history.action_text,
        task_history.action_date,task_history.user_id )
        SELECT ic.id, ic.task_id,username as user_id,action_text,action_date
        FROM inserted_comment ic
        LEFT JOIN users u ON ic.user_id = u.id;
    """
    )

After:

    query = text(
        """
        INSERT INTO task_history (
            project_id, task_id, action, action_text,
            action_date, user_id
        )
        VALUES (
            :project_id, :task_id, 'COMMENT', :comment_text,
            :current_date, :user_id
        )
        RETURNING
            task_history.id,
            task_history.task_id,
            (SELECT username FROM users WHERE id = task_history.user_id) AS user_id,
            task_history.action_text,
            task_history.action_date;
    """
    )

    # Define a dictionary with the parameter values
    params = {
        "project_id": comment.project_id,
        "task_id": comment.task_id,
        "comment_text": comment.comment,
        "current_date": currentdate,
        "user_id": user_data.id,
    }

spwoodcock · 2024-02-13T09:48:15Z

Also, ignore the failing backend test for this. I need to fix it on development for cases when the database is empty.

varun2948 · 2024-02-14T11:46:09Z

I just made some small updates:

Fixed the linting.
Improved the SQL readability with indenting & by removing the CTE from the insert query.
Used query params to insert vars, instead of f-strings. This is important to prevent possible SQL injection risk.

Before:

    query = text(
        f"""
        WITH inserted_comment AS (
        INSERT INTO task_history
        (project_id,task_id,"action",action_text,action_date,user_id)
        VALUES({comment.project_id},{comment.task_id},'COMMENT',
        '{comment.comment}','{currentdate}',{user_data.id})
        RETURNING
        task_history.id,task_history.task_id, task_history.action_text,
        task_history.action_date,task_history.user_id )
        SELECT ic.id, ic.task_id,username as user_id,action_text,action_date
        FROM inserted_comment ic
        LEFT JOIN users u ON ic.user_id = u.id;
    """
    )

After:

    query = text(
        """
        INSERT INTO task_history (
            project_id, task_id, action, action_text,
            action_date, user_id
        )
        VALUES (
            :project_id, :task_id, 'COMMENT', :comment_text,
            :current_date, :user_id
        )
        RETURNING
            task_history.id,
            task_history.task_id,
            (SELECT username FROM users WHERE id = task_history.user_id) AS user_id,
            task_history.action_text,
            task_history.action_date;
    """
    )

    # Define a dictionary with the parameter values
    params = {
        "project_id": comment.project_id,
        "task_id": comment.task_id,
        "comment_text": comment.comment,
        "current_date": currentdate,
        "user_id": user_data.id,
    }

I was wondering like how the liniting was working, if the sql code is unreadable. Now i got it !

spwoodcock · 2024-02-14T13:14:42Z

Oh no it wasn't actually to do with the SQL formatting.

It was some line length errors and redefined variables.

To run the linting locally and check, run:

pip install pre-commit
pre-commit run --all-files

varun2948 · 2024-02-14T13:17:18Z

i did that too. Does this command auto adjust the the sql query ?

spwoodcock · 2024-02-14T13:38:10Z

No it can't format multiline strings unfortunately.

It formats all other code though, sorts imports, lints code etc.

varun2948 · 2024-02-14T13:56:29Z

so we have to manually adjust the sql query right ?

spwoodcock · 2024-02-14T14:42:45Z

Yes exactly - any multiline strings need to be manually formatted.

One option is to copy the query into an SQL formatted online, then copy back

varun2948 · 2024-02-19T07:00:50Z

@spwoodcock, I just heard that Tm task history is huge on tm as it has alots of stuff related with tasks and due to lots of row it is taking alot of time to query data out of it. This is fine right ! the way we are replicating same as TM for the task history. Just a query…

spwoodcock · 2024-02-19T10:13:06Z

Yeah I can imagine!

It's probably not the most efficient or simple strategy implemented (using history entries for everything, including comments).

If we imagine that we have 100 projects, then 100 tasks per project, then 100 history records per task, this equals 1,000,000 task history entries quite quickly.

I'm not opposed to separating comments out to a different table: id, task id, user id, comment.

I guess I don't really have clarity on if we want to keep the structure of TM as much as possible for compatibility (what I have tried to do until now), or if we should optimise the FMTM schema to what we think is best.

Any thoughts @robsavoye ?

robsavoye · 2024-02-19T15:41:44Z

the task_history table is huge, and I don't think FMTM will have the same usage patterns. For TM, mappers interact with the website, but for FMTM, they're using Collect on their phone. TM has a project_chat table which I'm integrating into tm-admin, which might be more appropriate. FMTM will have some task_history itself, it's entirely possible an FMTM task will have bad data, so force TM to invalidate the task and have it mapped and validated. Then FMTM would have some changes when the task is field mapped again.

Since we're working on code sharing between FMTM and TM, we should make a unified plan. What type of chatting, commenting, user messaging, etc... do we need to support ? It's going to be different user stories, but the DB schema and access API should be the same. We'd implement this in tm-admin to use the new schema and the fully async code that uses it to avoid future performance problems.

* feat: project submissions page (#1150) * fix (routes): projectSubmissions route added * fix (icon): icon added * fix (projectSubmissions): setup * fix (projectSubmissions): styling adjustments * feat (projectSubmissions): projectInfo section slicing * feat (AssetModules): icon added * feat (taskSubmissions): taskSection - UI slicing * fix (projectSubmissions): UI fixes * feat (AssetModules): icons added * feat (select): className added to select * feat (projectSubmission): submissionsTable - filter section UI completed * feat: (AssetModules): icons added * fix (routes): normal import for projectSubmissions * feat (customTable): custom table added * feat (submissionsPage): table added to submission page * fix (taskSubmissions): mobile responsive * feat (recharts): recharts package added * icon added * feat (recharts): bar & pie chart added * feat (LineChart): lineChart added * feat (ProjectSubmissions): infographics - infographics slicing * feat (package): html2canvas package added * feat (submissionsPage): chartDownload - download feature added to the charts * fix (lineChart): line color changed * fix (UI): UI changes * fix (barChart): dynamic data for barChart * fix (pieChart): dynamic data for pieChart * fix (barChart): dynamic dataKey for xAxis * fix (lineChart): dynamic data for lineChart * fix (chart): bar/pie color changed * fix (lineChart): x & y axis labels added * feat (projectSubmissions): skeleton loader added * fix (projectSubmissions): filter padding increase * feat (projectSubmissions): skeletonLoader - skeletonLoader added to infographics and table * created api that returns the submission count by date * fix (projectSubmissions): infographicsCard - infographicsCard seperated to another component * feat (projectSubmissions): barChart - api fetch for formSubmissions * refactored response * feat: (projectSubmissions): submissionBar - dummy data replaced with actual data * fix (barChart): maxBarSize set * feat (projectSubmissions): task api call * feat (projectSubmissions): projectInfo - dynamic value for projTitle, tasks count, submission count * feat (projectSubmissions): taskSidebar - dynamic data added * feat (projectSubmissions): taskSidebar - zoomToTask feature added * Created Project dashboard * added organization info * Exception handling in s3 if no object found * refactor * fix: string based date to custom date format for over 7days * feat (projectSubmissions): contributorsTable - contributors table data fetch * fix (projectSubmissions): UI fix * fix (assetModules): duplicated icon removed * feat (projectSubmissions): projectInfo - api integration * fix (projectSubmissions): piechart - outerRadius adjusted * fix (projectSubmissions): message display if no data available * feat (projectSubmissions): taskLoading - skeleton loader added for tasks * fix (projectSubmissions): filter - UI fix * feat submissionsTable: api fetch for submissionsTable field & data * feat projectSubmissionsTable: dynamic header & data add with skeleton loader * fix projectSubmissions: relative path imports replaced with absolute path imports * feat projectSubmissionsTable: refresh table data functionality add * fix import: wrong validation import fix * feat submissionTable: mui tablePagination added * fix projectSubmissions: pagination api fetch * fix createProjectSlice: import fix * fix projectSubmissions: types added to slice * fix projectSubmissions: loading state to false if error arise in table api fetch * fix projectSubmissions: pagination disable on api fetch pending * fix select: optional type of null to value, errorMsg optional * fix projectSubmissions: table filter state added * fix lineChart: line color changed, xLabel & yLabel hidden if not present * fix projectSubmissions: validated vs mapped task lineChart add * fix bar/pieChart - color updated * fix projectSubmissions: redirection to submissionsPage on View Submissions click * fix projectSubmissions: loading state add on projectProgress api fetch * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: sujanadh <sujanadh07@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * fix loadingBar: loading bar moved to & imported from createnewproject folder * refactor createproject: old createProject folder deleted * refactor routes: removed import & removed old commented createproject routes * docs: add docs and task board badges to README * docs: update docs/task board badges to track development * fix: project creation draw and max boundary area (#1157) * fix uploadArea: areaValidation for fileUpload, draw hide if selectionType uploadFile, draw available if AOI reset * fix stepSwitcher: remove cursor-pointer on step hover * fix projctDetails: link to setting ODK server placed on Here text * refactor: add extra error handling if data extract download fails (#1158) * fix: downloading data extract accessing download_url key * add error log if data extract download fails * fix: project submission date and time formatted (#1160) * fix button: other btnType UI change * fix useOutsideClick: useOutsideClick hook add * feat projectDetails: buttons added including handleOutsideClick * feat projectOptions: options added for larger screens * fix projectDetails: redirect to projectSubmissions page on viewInfographics click * fix: zlib compress dynamic QR codes from frontend (#1159) * build(frontend): add pako for zlib compression, relock via pnpm * fix(frontend): add zlib compression to dynamic qrcodes * fix: zlib compression for qrcodes pako v2 (Uint8Array) * build: add qrcode conversion container to contrib * fix(frontend): remove svcfmtm from loginSlice initialState * fix(backend): append /v1/ to odk_token url for qrcodes * fix: map default odk credentials to organisations (#1123) * fix: add odk cred org schemas for db and pydantic * fix: add HttpUrlStr type for Url validation of str values * fix: fix OdkCentral schema to handle passwords, inherit * refactor: use new ODKCentralDecrypted, simplify proj create * fix: add org_deps get_org_odk_creds method * refactor(frontend): update create_project json structure * refactor(frontend): remove code to add #FMTM tag automatically * test: fix project route tests * refactor: revert setting outline_geojson in create_app_project * feat(frontend): option to add default odk creds for organisation * refactor(frontend): rename form during project creation --> category * refactor: remove additional route added during rebase * feat: set project delete endpoint to org_admin only * feat(frontend): project deletion capability * fix: organisation logo display full width (#1166) * fix homePage: exploreProjectCard image crop fix * fix splitTasks: text below progressBar remove * fix: create popup outside of async request (fix ios login) (#1167) * fix: baselayer url changed with token * feat: project admin role (#1133) * project_admin role deps * fix: default value for role in user_role table * feat: api to create new project manager * fix: linting errors * fix: linting errors * delete projects endpoint restricted for organisation admin only * fix:linting errors * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add new project admin allowed to project admin role * updated project admin role deps * feat: dependency org_admin to create project * feat: implemented access to project_admin to edit the project * feat: applied user dependency as per the requirement * project_admin role deps * fix: default value for role in user_role table * feat: api to create new project manager * fix: linting errors * fix: linting errors * delete projects endpoint restricted for organisation admin only * fix:linting errors * add new project admin allowed to project admin role * updated project admin role deps * feat: dependency org_admin to create project * feat: implemented access to project_admin to edit the project * feat: applied user dependency as per the requirement * update and refactor: user roles * refactor: simplify role access, combine into single query * refactor: update usage of roles, org_admin return dict of objects * fix: update project manager role usage * perf: add db indexes for org managers and user roles * dependency removed from validate_form endpoint * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * test: fix create_project pass url param * fix: add role for DEBUG enabled user * fix(frontend): pass org_id as url param to create_project * test: create test user account as ADMIN permission * fix: separate login logic from /me/ route * test: fix tests requiring auth permissions --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: sujanadh <sujanadh07@gmail.com> Co-authored-by: spwoodcock <sam.woodcock@protonmail.com> Co-authored-by: Sam <78538841+spwoodcock@users.noreply.github.com> * fix: organisation routes role usage & approval * fix: remove approved flag from organisations endpoint * feat: mapper role (#1163) * fix: remove approved flag from organisations endpoint * feat: Implement Mapper role * Refactor: check role from lower hierarchy and added role in end points * refactor: update mapper role to use check_access func * refactor: fix linting errors for backend --------- Co-authored-by: spwoodcock <sam.woodcock@protonmail.com> Co-authored-by: sujanadh <sujanadh07@gmail.com> * feat: manage / edit project UI (#1154) * fix (routes): projectSubmissions route added * fix (icon): icon added * fix (projectSubmissions): setup * fix (projectSubmissions): styling adjustments * feat (projectSubmissions): projectInfo section slicing * feat (AssetModules): icon added * feat (taskSubmissions): taskSection - UI slicing * fix (projectSubmissions): UI fixes * feat (AssetModules): icons added * feat (select): className added to select * feat (projectSubmission): submissionsTable - filter section UI completed * feat: (AssetModules): icons added * fix (routes): normal import for projectSubmissions * feat (customTable): custom table added * feat (submissionsPage): table added to submission page * fix (taskSubmissions): mobile responsive * feat (recharts): recharts package added * icon added * feat (recharts): bar & pie chart added * feat (LineChart): lineChart added * feat (ProjectSubmissions): infographics - infographics slicing * feat (package): html2canvas package added * feat (submissionsPage): chartDownload - download feature added to the charts * fix (lineChart): line color changed * fix (UI): UI changes * fix (barChart): dynamic data for barChart * fix (pieChart): dynamic data for pieChart * fix (barChart): dynamic dataKey for xAxis * fix (lineChart): dynamic data for lineChart * fix (chart): bar/pie color changed * fix (lineChart): x & y axis labels added * feat (projectSubmissions): skeleton loader added * fix (projectSubmissions): filter padding increase * feat (projectSubmissions): skeletonLoader - skeletonLoader added to infographics and table * created api that returns the submission count by date * fix (projectSubmissions): infographicsCard - infographicsCard seperated to another component * feat (projectSubmissions): barChart - api fetch for formSubmissions * refactored response * feat: (projectSubmissions): submissionBar - dummy data replaced with actual data * fix (barChart): maxBarSize set * feat (projectSubmissions): task api call * feat (projectSubmissions): projectInfo - dynamic value for projTitle, tasks count, submission count * feat (projectSubmissions): taskSidebar - dynamic data added * feat (projectSubmissions): taskSidebar - zoomToTask feature added * Created Project dashboard * added organization info * Exception handling in s3 if no object found * refactor * fix: string based date to custom date format for over 7days * feat (projectSubmissions): contributorsTable - contributors table data fetch * fix (projectSubmissions): UI fix * fix (assetModules): duplicated icon removed * feat (projectSubmissions): projectInfo - api integration * fix (projectSubmissions): piechart - outerRadius adjusted * fix (projectSubmissions): message display if no data available * feat (projectSubmissions): taskLoading - skeleton loader added for tasks * fix (projectSubmissions): filter - UI fix * feat (manageProject): manageProject added * feat (assetModules): icon added * feat (chips): chips added * fix (select): UI fix * feat (manageProject): partial UI completion for manage-project * feat (AssetModules): icon added * feat (UploadArea): custom upload area added * feat (manageProject): editTab - UI slicing completed for editTab * fix (AssetModules): duplicate icon removed * fix (select/manageProject): UI fix * feat (ManageProject): userTab - table added * fix (manageProject): mobile responsive fix * fix (customTable): css fix * feat (AssetModules): icons added * feat kebabMenu: kebabMenu added * feat (manageProject): table - kebab menu added to table * fix manageProject - UI fixes * fix (KebabMenu): added types * fix select: title background transparent * fix button: loading text checked first to display if button state loading * feat (manageProject): editProject - edit project description add * feat manageProject: formCategory list api fetch * feat uuid: uuid package add to generated ids * fix uploadArea: uuid add to uniquely identify each file * fix (manageProject): edit - projectId passes as prop to child * feat (manageProject): formUpdate - postFormUpdate api hit on update click * fix manageProject: individualProjectDetails fetch on parent component * fix navigation: navigation route added to manageProject and back btn * feat newProjectDetails: viewInfographics & manageProject btn add for mobile view under project options --------- Co-authored-by: sujanadh <sujanadh07@gmail.com> * ci: update all gh-workflows --> v1.4.7 * refactor: cast all fields for sqlalchemy with python types (#1173) * refactor: cast all fields for sqlalchemy with python types * fix: flatgeobuf conversion extract from Column first * feat: task history end point with associated user (#1071) * feat: separate task history end point * fix: add user relationship backref to task history * fix: add user details to task_history endpoint * refactor: revert to sqlalchemy backref names --------- Co-authored-by: sujanadh <sujanadh07@gmail.com> Co-authored-by: spwoodcock <sam.woodcock@protonmail.com> * feat(frontend): default organisation credentials during proj create (#1174) * fix: improve create organisation ui styling * fix: set org odk password field to password type * fix: allow for empty value for HttpUrlStr validation * feat(frontend): create project using default organization creds * refactor: update project creation comments --> tags * refactor: task status update endpoint to adjust mapper role (#1180) * update: task status update endpoint to adjust mapper role * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * fix: update db relations for generating submissions (#1179) * fix: update submission in s3 * Fix: error handling in s3 * removed optional from get_obj_from_bucket --------- Co-authored-by: sujanadh <sujanadh07@gmail.com> * fix uploadArea: geojson validation for uploaded file * feat: create / edit organization form (#1178) * feat createEditOrganization: routes/header setup * feat createEditOrganization: conditionally showing consent/createEdit forms * fix createEditOrganizationHeader: component renamed * feat consentQuestions: consent questions schema setup in constants * feat consentDetailsForm: leftSide container UI add * feat checkbox: custom radixUI checkbox component add * fix radioButton: className prop add to radioButton * fix consentQuestions: questions update * fix button: hover effect added to btn type other * fix checkBox: UI fix * feat consentDetailsForm: UI complete * feat consentDetailsForm: useForm integration & validations add * fix createEditOrganizationForm: organizationId props pass * fix RadioButton: required prop added * fix inputTextField: sublabel add * feat createEditOrganizationForm: form UI & validation complete * fix inputTextField: subLabel update * fix consentQuestions: redirection links added * fix organizationForm: post create organization detail * feat editOrganizationForm: individualOrganization api fetch * feat organizationForm: patch organizationData on update * fix organizationForm: naming change * fix organization: instructions updated & seperated to another component * fix manageOrganization: hide profile, org_type, osm_profile fields for edit organization * fix projectDetails: dowloadOptions open/close issue fix * feat approveOrganization: UI complete * fix organizationService: loading state false * fix createEditOrganizationForm: osm_profile remove from form * fix createEditOrganizationForm: filedType changed to password * feat userRole: userRole set to localStorage * fix manageOrganizations: UI fix, verified show/hide based on userRoles * fix manageOrganization: UI update * fix organizationDetailsValidation: osmProfile removed * fix createEditOrganization: clear consentForm state on organization creation * feat approveOrganization: approve api hit on verify click * refactor: org creation page consent questions (#1185) * fix consentQuestion: participated_in question required change to false * fix consentDetailsValidation: updated validation where user should mark atleast one checkbox for log_into question * feat: osm-rawdata for generating data extracts (#1183) * refactor: remove old unused DefineMapArea component * refactor: rename /generate endpoint --> /generate_project_data * fix(frontend): correctly send data_extract_type to project create * fix(frontend): add data_extract_url param, refactor DataExtract for clarity * build: add migration for projects.data_extract_url field * fix: add slash to osm-data-extract url for correct method * fix(frontend): pass through data extract url & type during proj creation * fix: set data extract url and type on generate endpoint * fix(backend): use osm-rawdata for data extract generation + filtering * fix(frontend): add form_category for osm data extract post params * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * build: update osm-rawdata --> v0.2.0 * build: update osm-rawdata --> v0.2.1 * feat: correctly set data extract url if custom upload or osm * fix: load fgb data extract and get geojson geoms * fix: accept xls forms in xls and xlsx format * fix: optional data extracts for tasking splitting algo * build: update fmtm-splitter --> v1.1.1 * refactor: simplify project file generation during creation * fix(frontend): passing params for tasking splitting algo * refactor: remove data extract type if osm generated * build: update minio browser to use env var * refactor: do not include minio port bind by default --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * test: fix create project generate_appuser_files usage * fix: project submission card UI enhancement (#1186) * feat assetModules: icon add * fix projectSubmissions: projectDataCard UI enhancement * fix: projectSubmissions: UI enhancement * fix projectSubmissions: infographics section mobile responsive * docs: use mdx_truly_sane_lists plugin for list ordering * fix: feature type not being saved as a geojson type * fix: axios interceptors fixing test * fix: axios interceptor testing * fix: project creation workflow fixes (task splitting) (#1194) * fix: remove Field(Form()) from schemas * refactor: move geometry generation to project_schemas * fix(frontend): send project outline geojson during create * fix: only send custom data extracts to splitter * fix: optional project_task_name in schema * refactor(frontend): remove refs to project_task_name * refactor(backend): remove extract_type from data-extract-url * fix(frontend): setting data extract url if osm generated extract * test: fix or remove tests during project creation * fix: add odk_central_url to org details returned * feat: add community_type for organisations, add unapproved org list endpoint (#1197) * create organisation permission updated to login_required * api to list unapproved organisations * Feat: Added email and community_type field in organisation * fix: changed org_id to mandatory to create project * fix: added organisation_id in payload of create project * fix: await check crs, added email and community type in test organisation * refactor: proper enum field community type in test organisation * build: update community_type migration number & logic * refactor: remove email field from organisation * refactor: remove organisation_id from ProjectUpload model * refactor: remove email from dborg model for conftest * refactor: remove organisation_id from create_project POST json * test: fix remove organisation_id from ProjectUpload * fix: add optional organisation_id to ProjectUpload * feat: add project to org_user_dict if present * fix: extract project from org_user_dict on deletion * test: fix tests to include organisation_id extracted from fixture --------- Co-authored-by: Niraj Adhikari <nrjadkry@gmail.com> Co-authored-by: sujanadh <sujanadh07@gmail.com> Co-authored-by: spwoodcock <sam.woodcock@protonmail.com> * fix: org creation using Form params * feat: endpoints for task comments (#1171) * feat: taskcomment model added * feat: crud task comment * feat: task comment crud api route added * feat: task comment schema added * fix: removed use of dictionary to object * fix: change row name by id with task_comment_id * fix: renamed comment_task_id_to_id * feat: migration add-task-comment * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * renamed task_comment index * fix: removed new table migration for taskcomment * feat: used task_history table to get task_comment and add task_comment Using task_history table maintained action row as COMMENT and added comment * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix: linting issue * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * refactor: improve sql, use query params, fix linting errors * refactor: minor code cleanup bundled in merge --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: spwoodcock <sam.woodcock@protonmail.com> * fix: update task status when mapping starts (#1208) Co-authored-by: sujanadh <sujanadh07@gmail.com> * feat: init public beta org and svcfmtm user migration (#1206) * build: remove odk vars from frontend build dev compose * build: remove user password field from init schema * build: fix only continue migration if sql script succeeds * build: update migration entrypoint envsubst + error handling * build: migration to create fmtm public beta org + svcfmtm * build: add gettext for envsubst command in dockerfile * build: use odk user email for svcfmtm user email * fix: replace org init with startup code for pass encrypt * fix: specify user id for svcfmtm admin init * build: fix remove sequential user id (create manually) * build: remove default user creation from migration script * build: fix project-roles migration if enum exists * fix (manageProject) - formUpdate: remove formCategory dropdown for update * fix (projectSubmissions): taskList: search functionality add to task-list section * fix taskIdSearch: message display if searched taskId not present * fix:added interceptor for cookie issue * feat: filters on submission table (#1191) Co-authored-by: sujanadh <sujanadh07@gmail.com> * build: increase proxy_read_timeout for longer connection time * refactor: remove old edit-project code (#1210) * refactor (frontend): rename changeset to hashtags (#1211) * fix editProject: fieldName changeset change to tags, tags field required set to false * fix namingChange: field title Tags changed to Hashtags * feat: api returning details of unapproved org (#1218) Co-authored-by: sujanadh <sujanadh07@gmail.com> * feat: improve submissions dynamic filtering (#1217) * fix submissionTable: filterObj key name change, reviewStateData constant add * feat: filters on submission table * fix inputTextField: input label fix * fix submissionTable: submitted by textField add * feat customDatePicker: react-datepicker add * fix projectSubmissions: datepicker add & payload add for api fetch * fix projectSubmissions: submissionFilters UI enhancement * fix projectSubmissions: toggle infographics/table view UI re-arrangement * feat pieChart: legend add * fix submissionInfographics: dynamic value for projectProgress pieChart * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: sujanadh <sujanadh07@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * fix: data extract generation and task splitting with polylines (#1219) * build: bump osm-rawdata & fmtm-splitter versions for fixes * fix: use useStWithin=False for data extract generation with polyline * fix: first coordinate check when check_crs geojson * Updated osm-fieldwork version (#1222) * fix: submissions form fields * refactor central crud * pdm lock file updated --------- Co-authored-by: sujanadh <sujanadh07@gmail.com> Co-authored-by: Niraj Adhikari <nrjadkry@gmail.com> * feat (frontend): approve organization api integration (#1215) * fix managProject: UI fix * feat manageOrganization: redirect to approveOrganization on org click on To Be Verified tab org list * fix rejectOrganization: service for rejectOrganization add * feat organizationApproval: state management to show approving/rejecting status, redirect to manage-org after approval success * fix: update schema to edit only required fields (#1223) Co-authored-by: sujanadh <sujanadh07@gmail.com> * Instructions in create and edit project (#1224) * fix: update schema to edit only required fields * feat: added instructions in project create and edit --------- Co-authored-by: sujanadh <sujanadh07@gmail.com> * fix editProject: formUpdate file extension spelling corrected to xlsx * fix header: activeTab selection based on url pathname * refactor primaryAppBar: unnecessary commented code remove * feat: added approved in get organisation response (#1226) Co-authored-by: sujanadh <sujanadh07@gmail.com> * feat assetModules: tuneIcon add * feat shadcnDropdown: shadcn dropdown add * fix select: zIndex add to selectContent * fix submissionsFilter: UI enhancement * fix submissionTable: logic add to handle Point & object to display dynamic table * feat submissionsTable: buttons add * fix modal: modal props type add * fix taskService: josmService comment update * feat projectSubmissions: uploadToJosm api integration & modalOpen on josmError * fix: removed deps from delele org api (#1233) Co-authored-by: sujanadh <sujanadh07@gmail.com> * fix dropdown: dropdownMenu UI fix * feat submissionTable: download as csv & json dropdown add on download option * fix taskService: axios get replace fetch for convertXMLToJOSM service * fix: required odk credentials if no organisation default (#1205) * fix ICreateProject: projectDetailsTypes fix * fix projectDetialsForm: default of value #FMTM add to tags field * fix projectDetailsForm: odk url, email, password made required * fix projectDetailsForm: default FMTM tag removal * fix checkbox: className props add * fix createEditOrganization: remove email field, checkbox add to conditionally show ODK credentials field * fix organizationForm: remove email field * fix customDatePicker: types add * fix createEditOrganizationFrom: ODK credential fields optional/required based on checkbox * feat createNewProject: conditionally hide/unhide ODK credentials fields based on checkbox selection * fix createNewProject: clear odkCredentials on defaultODKCredentials checkbox uncheck * fix createNewProject - step1: UI enhancement, splitted form updated to single column form * fix organizationForm: getUnapprovedOrganization api replacement * fix unapprovedOrg: return obj instead of array * fix organization: state not clear issue solve * fix organization: editOrganization organizationFormData state clear * fix createEditOrganization: comments add * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * feat (frontend,backend): my organisations tab implemented (#1227) * feat: my organisations api route added * feat: get my organisation function * feat: added get organisation api service * feat: organisationGridCard Component created * feat: Type added to organisation slice * Feat: My organisation tab for data fetching * refactor: remove missed print statement --------- Co-authored-by: spwoodcock <sam.woodcock@protonmail.com> * refactor: improve flatgeobuf loading and generation (#1235) * refactor: update flatgeobuf --> geojson to extract properties * fix: create flatgeobuf files with spatial index by default * docs: comment note for future fgb generation * docs: extra info on fgb generation with tags * fix: project org UI issues (#1230) * fix editProject: formUpdate file extension spelling corrected to xlsx * fix header: activeTab selection based on url pathname * refactor primaryAppBar: unnecessary commented code remove * feat assetModules: tuneIcon add * feat shadcnDropdown: shadcn dropdown add * fix select: zIndex add to selectContent * fix submissionsFilter: UI enhancement * fix submissionTable: logic add to handle Point & object to display dynamic table * feat submissionsTable: buttons add * fix modal: modal props type add * fix taskService: josmService comment update * feat projectSubmissions: uploadToJosm api integration & modalOpen on josmError * fix dropdown: dropdownMenu UI fix * feat submissionTable: download as csv & json dropdown add on download option * fix taskService: axios get replace fetch for convertXMLToJOSM service * test (frontend): case config, add test cases (#1231) * feat: Test cases configuration changes for vitest * feat(Test): Button test cases added with docstring * feat(Testcases): added a small test cases to check if test is working * fix: add created_by user id to organisation table (#1232) * feat: added user in organisation * feat: make user org manager after org approval * fix: added user_id in org pytest fixture * refactor: simplify organisation user_id --> created_by int field * build: update migrations to match org db model * refactor: simplify org approval logic * refactor: bullet point in migration file --------- Co-authored-by: sujanadh <sujanadh07@gmail.com> Co-authored-by: spwoodcock <sam.woodcock@protonmail.com> * feat submissionsPage: redirect to respective submissions & task submissions page on button click * Feat verification tag add (#1234) * feat organizationList: verification tag add on organization card * fix projectTaskStatus: enum value display * fix manageOrganization: hide new btn if no token & show verified/unverified tab on ALL tab selection only * fix manageOrganization: add verification tag & redirect to approve org if org not verified * fix: qr popup on task status & styling (#1184) * fix (environment): tasksStatus - actionKey name change, btnBG key add * fix assetModules: icon add * fix button: UI change, id prop inclusion * fix qrCodeComponent: UI enhancement, component moved from projectDetails to taskSectionPopup * fix dialogTaskActions: UI enhancements, buttons add * fix qrCodeComponent: qrcode access form parent component, code cleanup, UI enhancement * fix qrCodeComponent: UI fix * fix projectdetails: old projectDetails page replaced with new * fix button: id prop relaced with btnId prop * fix qrPopup: btnid extract from event.currentTarget * fix projectDetails: navigate path to project_details fix * feat filterParams: filter out null objects * feat projectSubmissions: searchParams add on individual submissions page navigation * fix projectSubmissions: navigate to projectSubmissions with searchParams * feat: added bbox in read project's outline (#1244) Co-authored-by: sujanadh <sujanadh07@gmail.com> * feat: flatgeobuf data extracts on frontend & backend (#1241) * refactor: add data_extract_url to ProjectOut schema * feat: load remote flatgeobuf data extracts from S3 * fix(frontend): correctly load nested fgb GeometryCollection type * feat: read/write flatgeobuf, split geoms by task in database * feat: split fgb extract by t ask, generate geojson form media * refactor: deletion comment for frontend files not required * build: remove features table from db * feat: allow uploading of custom data extracts in fgb format * test: update tests to use flatgeobuf data extracts * refactor: rename var for clarity * ci: minify backend test data + ignore from prettier * fix: move OdkDecrypted logic into project_deps (#1239) * fix:commit features to db before generating task files * refactor: convert repeated odk_cred into a function * fix: reuse DbProject object when retrieving odk creds --------- Co-authored-by: sujanadh <sujanadh07@gmail.com> Co-authored-by: spwoodcock <sam.woodcock@protonmail.com> * fix: comment broken SubmissionMap import on frontend * refactor: changed sync to async gather_all_submission_csvs (#1258) Co-authored-by: sujanadh <sujanadh07@gmail.com> * feat(manage-organizations): add `OrganizationCardSkeleton` component * feat(manage-organisations): implement skeleton loader while Organizations are being fetched * refactor: reorganize the code to make it cleaner * fix: send bbox from geojson properties to josm (#1256) * build: upgrade osm-rawdata --> 0.2.3 & relock all versions * fix: file validity check on fileUpload only once (#1262) * fix: use raw sql for organisation crud (#1253) * fix: update initial_feature_count --> feature_count and populate values (#1265) * build: rename tasks.initial_feature_count --> tasks.feature_count * refactor: rename initial_feauture_count in endpoints * fix: dynamic submissions legend chloropeth (#1250) * fix projectSubmissions: dynamic mapLegend and taskChloropeth on increasing submissions count * feat projectSubmissionsMap: popup showing expected/submissions count add * fix homeSCSS: width fix * fix popupHeader: border fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * refactor: removing redundant code for removing extra closing </osm> tag (#1264) Co-authored-by: sujanadh <sujanadh07@gmail.com> * fix: task splitting with custom data extract (#1255) * fix: handle GeometryCollection wrappers in geojson parse * fix: task splitting when custom data extract uploaded * feat: add map popups to data extract geometries (#1266) * refactor: always return all properties on map click * refactor: rename submissionModel --> taskModel * refactor: configurable primary property key for map AsyncPopup * feat: add map popup on data extract geom click * build: make initial_feature_count migration idempotent --------- Co-authored-by: Nishit Suwal <81785002+NSUWAL123@users.noreply.github.com> Co-authored-by: sujanadh <sujanadh07@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: NSUWAL123 <suwalnishit@gmail.com> Co-authored-by: Deepak Pradhan (Varun) <37866666+varun2948@users.noreply.github.com> Co-authored-by: Deepak Pradhan <deepak.pradhan048@gmail.com> Co-authored-by: Niraj Adhikari <41701707+nrjadkry@users.noreply.github.com> Co-authored-by: Sujan Adhikari <109404840+Sujanadh@users.noreply.github.com> Co-authored-by: Niraj Adhikari <nrjadkry@gmail.com> Co-authored-by: prajwalism <prajwalkhadgi11@gmail.com>

varun2948 added 10 commits January 17, 2024 11:33

feat: taskcomment model added

9827495

feat: crud task comment

9c54907

feat: task comment crud api route added

d1142a9

feat: task comment schema added

df2141c

Merge branch 'development' of github.com:hotosm/fmtm into feature/tas…

5f423eb

…k_comment

Merge branch 'development' of github.com:hotosm/fmtm into feature/tas…

931004a

…k_comment

fix: removed use of dictionary to object

bc41b8a

fix: change row name by id with task_comment_id

e646b4a

fix: renamed comment_task_id_to_id

9142cc9

feat: migration add-task-comment

fd540d0

varun2948 temporarily deployed to test February 6, 2024 10:36 — with GitHub Actions Inactive

varun2948 requested a review from spwoodcock February 6, 2024 10:36

github-actions bot added backend Related to backend code migration Contains a DB migration labels Feb 6, 2024

[pre-commit.ci] auto fixes from pre-commit.com hooks

89f0b7f

for more information, see https://pre-commit.ci

pre-commit-ci bot temporarily deployed to test February 6, 2024 10:36 Inactive

varun2948 requested a review from nrjadkry February 6, 2024 10:36

pre-commit-ci bot had a problem deploying to test February 6, 2024 10:36 Failure

varun2948 added 3 commits February 6, 2024 16:43

renamed task_comment index

45845a6

Merge branch 'development' of github.com:hotosm/fmtm into feature/tas…

af88180

…k_comment

Merge branch 'feature/task_comment' of github.com:hotosm/fmtm into fe…

3033ca1

…ature/task_comment

varun2948 temporarily deployed to test February 6, 2024 11:00 — with GitHub Actions Inactive

varun2948 temporarily deployed to 1171/merge February 6, 2024 11:00 — with GitHub Actions Inactive

varun2948 had a problem deploying to test February 6, 2024 11:03 — with GitHub Actions Failure

spwoodcock approved these changes Feb 13, 2024

View reviewed changes

src/backend/app/db/db_models.py Show resolved Hide resolved

src/backend/app/tasks/tasks_routes.py Outdated Show resolved Hide resolved

Merge branch 'development' into feature/task_comment

8a5f26b

spwoodcock temporarily deployed to test February 13, 2024 09:29 — with GitHub Actions Inactive

spwoodcock had a problem deploying to test February 13, 2024 09:30 — with GitHub Actions Failure

spwoodcock marked this pull request as ready for review February 13, 2024 09:30

nrjadkry approved these changes Feb 13, 2024

View reviewed changes

refactor: improve sql, use query params, fix linting errors

5a950f8

spwoodcock temporarily deployed to test February 13, 2024 09:44 — with GitHub Actions Inactive

spwoodcock had a problem deploying to test February 13, 2024 09:44 — with GitHub Actions Failure

spwoodcock temporarily deployed to test February 13, 2024 10:01 — with GitHub Actions Inactive

spwoodcock temporarily deployed to 1171/merge February 13, 2024 10:01 — with GitHub Actions Inactive

spwoodcock temporarily deployed to test February 13, 2024 10:04 — with GitHub Actions Inactive

refactor: minor code cleanup bundled in merge

85d6ad2

spwoodcock temporarily deployed to test February 13, 2024 10:32 — with GitHub Actions Inactive

spwoodcock merged commit af1b9ee into development Feb 14, 2024
5 checks passed

spwoodcock deleted the feature/task_comment branch February 14, 2024 07:42

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Endpoints for task comments #1171

Endpoints for task comments #1171

varun2948 commented Feb 6, 2024

spwoodcock commented Feb 6, 2024 •

edited

Loading

varun2948 commented Feb 6, 2024

spwoodcock commented Feb 6, 2024

varun2948 commented Feb 6, 2024

spwoodcock commented Feb 6, 2024 •

edited

Loading

varun2948 commented Feb 6, 2024

varun2948 commented Feb 13, 2024

spwoodcock left a comment •

edited

Loading

spwoodcock commented Feb 13, 2024 •

edited

Loading

spwoodcock commented Feb 13, 2024

varun2948 commented Feb 14, 2024

spwoodcock commented Feb 14, 2024

varun2948 commented Feb 14, 2024

spwoodcock commented Feb 14, 2024

varun2948 commented Feb 14, 2024

spwoodcock commented Feb 14, 2024 •

edited

Loading

varun2948 commented Feb 19, 2024 •

edited

Loading

spwoodcock commented Feb 19, 2024

robsavoye commented Feb 19, 2024

Endpoints for task comments #1171

Endpoints for task comments #1171

Conversation

varun2948 commented Feb 6, 2024

What type of PR is this? (check all applicable)

Related Issue

Describe this PR

Screenshots

Checklist before requesting a review

[optional] What gif best describes this PR or how it makes you feel?

spwoodcock commented Feb 6, 2024 • edited Loading

varun2948 commented Feb 6, 2024

spwoodcock commented Feb 6, 2024

varun2948 commented Feb 6, 2024

spwoodcock commented Feb 6, 2024 • edited Loading

varun2948 commented Feb 6, 2024

varun2948 commented Feb 13, 2024

spwoodcock left a comment • edited Loading

Choose a reason for hiding this comment

spwoodcock commented Feb 13, 2024 • edited Loading

spwoodcock commented Feb 13, 2024

varun2948 commented Feb 14, 2024

spwoodcock commented Feb 14, 2024

varun2948 commented Feb 14, 2024

spwoodcock commented Feb 14, 2024

varun2948 commented Feb 14, 2024

spwoodcock commented Feb 14, 2024 • edited Loading

varun2948 commented Feb 19, 2024 • edited Loading

spwoodcock commented Feb 19, 2024

robsavoye commented Feb 19, 2024

spwoodcock commented Feb 6, 2024 •

edited

Loading

spwoodcock commented Feb 6, 2024 •

edited

Loading

spwoodcock left a comment •

edited

Loading

spwoodcock commented Feb 13, 2024 •

edited

Loading

spwoodcock commented Feb 14, 2024 •

edited

Loading

varun2948 commented Feb 19, 2024 •

edited

Loading